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DETAILED ACTION 



Claim Rejections - 35 USC §112 

1 . The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 1- 16 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

3. Regarding claim 1 , the scope of the claim is unclear as to whether the claim is 
attempting to claim a method or an apparatus. The claim should recite one or the other. 

4. Claim 8 recites the limitation "the color model" in line 1. There is insufficient 
antecedent basis for this limitation in the claim. 

5. The remaining claims are rejected because they are dependent on another claim 
that is rejected by 35 USC 112. 

Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 
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7. Claims 1, 3, 5-7, and 9-14 are rejected under 35 U.S.C. 102(e) as being 
unpatentable by U.S. Patent No. 6985620 (Sawhney et al). 

8. Regarding claim 1 , Sawhney et al discloses a robust apparatus and method for 
visually tracking an object in three dimensions (fig. 4, and the apparatus that carries out 
fig. 4), comprising steps of (a) capturing the object with two or more imaging sensors 
(fig. 1 , items 108); (b) maintaining a large number of 3D target location hypotheses 
shown by steps 400 and 404 in fig. 4, since there are many relevant features that are 
hypotheses of the 3D target location of the 3D model (fig. 3, solid lines); (c) projecting 
each target location hypothesis from 3D space to 2D image spaces of said imaging 
sensors, since the 3D relevant features are projected into the estimated pose of the 
camera, a 2D image (fig. 4, item 406); (d) measuring in images of captured said imaging 
sensors, confidences about the presence of said target object, since the error is 
measured (fig. 4, item 410) which is directly related to the confidence since the more 
error there is, the less confidence there is; and (e) combining said measured 
confidences of said hypotheses, since all the errors are used (fig. 4, items 412) to 
obtain 3D location of said target object, the updated pose. 

9. Regarding claim 3, Sawhney et al discloses that the apparatus is a computer, 
comprising a video capture apparatus (fig. 1, items 108 and col. 4, line 19), memory, 
such as optical storage media (col. 15, line 29) and a processor, that which processes 
and performs the functions of a computer readable media (col. 15, lines 24-27). 

10. Regarding claim 5, Sawhney et al discloses that the projecting target locations 
(fig. 4, item 406) is performed with projections that are obtained by calibrating said video 



Application/Control Number: 1 0/61 8,393 Page 4 

Art Unit: 2624 

sensors with respect to a reference coordinate system, since the correct range is 
determined for the video sensors based on the estimated pose (fig. 4, item 402) in order 
to perform the projections with respect to a reference coordinate system, the 3D 
coordinate system of the model (fig. 4, item 400). 

1 1 . Regarding claim 6, Sawhney et al discloses measuring confidences is performed 
based on color when disclosing that a color model can be used in fig. 4, step 404 (col. 
5, lines 30-31) and motion cues when disclosing that confidences are measured after 
the step of using the motion cue of a previous frame in step 402 of fig. 4 (col. 4, lines 
64-67). 

12. Regarding claim 7, a color model is used to calculate color cues (col. 5, lines 30- 
31) 

13. Regarding claim 9, Sawhney et al discloses motion cues are calculated by 
calculating differences between images captured sequentially by the said imaging 
sensors since an assumed motion is found and can be used to find the next frame, and 
since optical flow can be used to estimate the estimated pose (col. 4, line 64- col. 5, line 

8). 

14. Regarding claim 10, Sawhney et al discloses that maintaining 3D target location 
hypothesis is performed by creating at each time step, since this method can be used 
for a sequence of frames (col. 3, line 64) a set of 3D target location hypotheses (fig. 4, 
item 404 and fig. 3, solid lines). 
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1 5. Regarding claim 1 1 , Sawhney et al discloses that the 3D target location 
hypotheses are created based on known 3D target location hypotheses from a previous 
time step (col. 4, lines 64-65). 

16. Regarding claim 12, Sawhney et al discloses 3D target locations are initially 
distributed randomly in the space viewed by said imaging sensors, when interpreting the 
3D target locations as all the locations of the model, as seen in fig. 2 (step 400, fig. 4), 
all distributed randomly in space. Later, the target locations are reduced to only the 
relevant features in fig. 3 (fig. 4, step 404). 

17. Regarding claim 13, maintaining 3D target location hypotheses involves adding 
random displacements to said location hypothesis at each time step, displacements 
estimated by optical flow and RANSAC (col. 5, lines 2-18). 

18. Regarding claim 14, Sawhney et al discloses that combining said measurements 
involve calculating the average of the locations of said 3D target location hypotheses, 
since the location of the 3D target locations is the same pose, thus, the average of all 
the hypotheses is the pose. At step 412 of fig. 4, the combining matching errors involve 
the calculated location, or pose. 

Claim Rejections - 35 USC § 103 

19. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

» 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Application/Control Number: 10/618,393 Page 6 

Art Unit: 2624 

20. Claims 2 and 4 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Sawhney et al in view of U.S. Patent No. 7106885 (Osterwell). 

Regarding claim 2, Sawhney et al discloses all of the claimed elements as set 
forth above and incorporated herein by reference. Sawhney et al further discloses 
using color models and color in the system (coL 5, lines 30-31). 

Sawhney et al does not disclose expressly that the imaging sensors are color 
cameras. 

Osterwell discloses capturing images in color for later use in processing, thus the 
image sensors are color cameras (col. 2, lines 46-50). 

Sawhney et al and Osterwell are combinable because they are from the same 
field of endeavor, i.e. evaluating image positioning (Osterwell, col. 1, lines 15-16) 

At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to use a color camera to gain color information for later processing. 

The suggestion/motivation for doing so would have been to get the quickest, 
most accurate representation of the object by directly providing color information from 
the camera instead of trying to derive the color in some other manner from a non-color 
image. 

Therefore, it would have been obvious to combine the method and apparatus of 
Sawhney et al with the color camera of Osterwell to obtain the invention as specified in 
claim 2. 

21. Regarding claim 4, Osterwell discloses using IEEE1394 (col. 12, lines 4-5). 
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22. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Sawhney 
et al in view of U.S. Patent No. 6658136 (Brumitt). 

Sawhney et al discloses all of the claimed elements as set forth above and 
incorporated herein by reference. Sawhney et al further discloses a color model is used 
to calculate color cues (col. 5, lines 30-31). 

Sawhney et al does not disclose expressly the color model of the target is 
represented by a histogram that is estimated by collecting color samples of the target 
object 

Brumitt discloses that color histograms are used to compare characterizations of 
objects (col. 3, lines 10-16). 

Sawhney et al and Brumitt are combinable because they are from the same field 
of endeavor, i.e. tracking images (Brumitt, col. 1, lines 9-10). 

At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to use a color histogram to represent the color model. 

The suggestion/motivation for doing so would have been to provide an accurate 
way to characterize an image that can be easily compared. 

Therefore, it would have been obvious to combine the apparatus and method 
Sawhney et al with the color histogram model of Brumitt to obtain the invention as 
specified in claim 8. 

23. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Sawhney et al in view of U.S. Patent No. 6072903 (Maki et al). 
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Regarding claim 15, Sawhney et al discloses all of the claimed elements as set 
forth above and incorporated herein by reference. 

Sawhney et al does not disclose expressly target is a human appendage. 

Maki et al discloses a target to be tracked is a human appendage, a head (col. 1 , 
lines 11-12). 

< 

Sawhney et al and Maki et al are combinable because they are from the same 
field of endeavor, i.e. tracking objects (Maki et al, col. 1, lines 11-12). 

At the time of the invention, it would have been obvious to a person of ordinary 
skill in the art to track a human appendage such as the head. 

The suggestion/motivation for doing so would have been to provide a more 
flexible system by providing additional objects to be tracked instead of simply the 
shapes discloses by Sawhney et al as an example, and to provide a means for 
personnel identification by finding the head which is a common way to identify a person. 

Therefore, it would have been obvious to combine the apparatus and method of 
Sawhney et al with the tracking of a human appendage as disclosed by Maki et al to 
obtain the invention as specified in claim 15. 

24. Regarding claim 16, Maki et al discloses that the target is a human head (col. 1, 
lines 11-12). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kathleen S. Yuan whose telephone number is (571)272- 
2902. The examiner can normally be reached on Monday to Thursdays, 9 AM to 5 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Mancuso can be reached on (571)272-7695. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

KY 

11/27/2006 
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